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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )D Responsive to communication(s) filed on . 

2a)S This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) S Claim(s) 1-11 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) Q Claim(s) is/are allowed. 

6) S Claim(s) 1-11 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)E3 The drawing(s) filed on 02 May 2002 is/are: a)£3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)[3 All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. ^ Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

1 . This Office Action is in response to an AMENDMENT entered September 28, 
2004 for the patent application 1 0/049,650 filed on May 2, 2002. 

2. The First Office Action of March 25, 2004 is fully incorporated into this Final 
Office Action by reference. 



Status of Claims 

3. Claims 1-1 1 are amended. Claims 1-1 1 are pending. 



Claims Objections 

4. Claim 5 is objected to under 37 CFR 1 .75(c) as being in improper form because 
a multiple dependent claim should refer to other claims in the alternative only. While one 
use in a claim of the phase "either of actions a3) and b3)" would be acceptable, multiple 
usage of the same phrase violates the concept of reference in the alternative only. See 
MPEP § 608.01 (n). Accordingly, the claim 5 has not been further treated on the merits. 
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Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

6. Claims 1-1 1 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Mehrotra et al (MIT Press, 1997, referred to as Mehrotra). 

Claim 1 

Mehrotra anticipates identifying and eliminating synapses that have no significant 
influence on the curve of the risk function (Mehrotra, p 116, 1 8-19; p 118, 1 4-5; 
Examiner's Note (EN): para 15 applies; curve of the risk function is synonymous with 
Mehrotra's error analysis), including a1) selecting first and second sending-neurons that 
are connected to the same receiving neuron by respective first and second synapses 
(Mehrotra, p, 19, Fig. 1.14; p 1 16, 1 31-32), a2) assuming a correlation of response 
signals from said first and second sending neurons to the same receiving neuron 
(Mehrotra, p 19, Fig. 1.14; p 46, Fig. 2.5), a3) interrupting the first synapse and 
adapting in its place a weight of the second synapse (Mehrotra, p 19, Fig. 1 .14; p 1 17, 
Fig. 4.6; EN: the weight of the second synapse is the result of the removal of the first 
synapse), a4) comparing the output signals of the neural network changed in 
accordance with action a3) with the output signals of the unchanged neural network 
(Mehrotra, p 19, Fig. 1.14; p 116. 1 33-34; p 117, Fig. 4.6), and a5) if the comparison 
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result does not exceed a predetermined level, eliminating the first synapse thereby 
simplifying the structure of the neural network (Mehrotra, p 19, Fig. 1.14; p 118, 1 1-5). 
Claim 2 

Mehrotra anticipates wherein the first and second selected sending neurons are 
located on the same layer (Mehrotra, p 19, Fig. 1.14). 
Claim 3 

Mehrotra anticipates further comprising adapting a value of a bias of the 
receiving neuron in action a3) (Mehrotra, p 19, Fig. 1.14; p 116, 1 33-34, p117, 1 10-11). 
Claim 4 

Mehrotra anticipates identifying synapses that have no significant influence on 
the curve of the risk function further includes b1 ) selecting a synapse (Mehrotra, p 1 17, 
1 1-9; EN: para 15 applies; curve of the risk function is synonymous with Mehrotra's 
error analysis), b2) assuming that said selected synapse does not have a significant 
influence on the curve of the risk function (Mehrotra, p 117, 1 1-9), b3) interrupting 
selected said synapse (Mehrotra, p 117, 1 1-9), b4) comparing the output signals of the 
neural network changed in accordance with action b3) with the output signals of the 
unchanged neural network (Mehrotra, p 117, 1 1-9), and b5) if the comparison result 
does not exceed a predetermined level, eliminating the selected synapse, further 
simplifying the structure of the neural network (Mehrotra, p 1 17, 1 1-9). 
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Claim 6 

Mehrotra anticipates calculating a value of a likelihood function for the neural 
network to represent the expected output of the neural network (Mehrotra, p 1 18, 1 1-5; 
EN: para 15 applies; Mehrorta' AE is synonymous with a likelihood function). 
Claim 7 

Mehrotra anticipates comparing structure variants of the neural network using a 
significance test (Mehrotra, p 118, 1 1-5; EN: para 15 applies; Mehrorta' AE check 
against a threshold is a significance test). 
Claim 8 

Mehrotra anticipates the structure variants of the neural network are compared 
using a CHI-SQUARED test (Mehrotra, p 118, 11-5; EN: para 15 applies; CHI- 
SQUARED test is a measure of the deviation from the expectation as is Mehrorta' AE 
check against a threshold). 
Claim 9 

Mehrotra anticipates structure variants of the neural network are compared 
using a BOOT-STRAPPING method (Mehrotra, p 117, 1 1-9; EN: para 15 applies; 
BOOT-STRAPPING is a sampling method for estimating values that would be difficult to 
infer analytically; Mehrorta check against a threshold functions in a synonymous way). 
Claim 10 

Mehrotra anticipates that, to compare two structure variants of the neural 
network includes calculating the ratio of the values of the likelihood functions for said 
two structure variants (Mehrotra, p 1 18, 1 1-5; EN: para 15 applies; a ratio test will vary 
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from zero to infinity with unity as the point of equality; similarly, Mehrorta's AE check 
against a threshold will accordingly vary in a similar fashion where the threshold is 
equivalent to the applicants unity value). 
Claim 11 

Mehrotra anticipates optimizing the strengths of the connections between the 
neurons, according to a simplex method (Mehrotra, p 117, 1 1-9; EN: para 15 applies; 
Mehrotra's selective node removal represents optimization of the Mehrotra's initial 
network since Mehrotra's goal is a pruned (optimized); Mehrotra's hyperplane 
adjustments made in Fig. 4.6 are synonymous to the simplex method of the inventor). 



Response to Arguments 

7. The abstract objection under 37 CFR 1 .72(b) is withdrawn. 

8. The objection to the specification under MPEP 608.01 (a) is withdrawn. 

9. The rejections of claims 1-1 1 under 35 USC 101 are withdrawn. 

10. The rejections of claims 1-1 1 under 35 USC 112, first paragraph, are withdrawn. 

1 1 . The rejections of claims 1-1 1 under 35 USC 1 12, second paragraph, are 
withdrawn. 

1 2. Applicant's arguments filed on September 28, 2004 related to Claims 1 -1 1 have 
been fully considered but are not persuasive. 

In reference to Applicant's argument: 
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Mehrotra et al. disclose the structure of a neural network in Fig. 1.14, as noted by the examiner. On page 
116, Mehrotra et al. also disclose "pruning" the network, that is, eliminating nodes and connections that 
are considered unimportant. In Fig. 4.6, Mehrotra et al. describe a generic pruning algorithm, by which 
nodes or connections are deleted is the removal does not penalize performance beyond a desirable 
tolerance level. Mehrotra et al. then outline, on pages 116-118, four procedures that can be used to 
implement the network pruning methodology. None of these procedures includes selecting first and 
second sending neurons that are connected to the same receiving neuron by respective first and second 
synapses, assuming a correlation of response signals from said first and second sending neurons to the 
same receiving neuron, interrupting the first synapse and adapting in its place a weight of the second 
synapse, comparing the output signals of the changed neural network with the output signals of the 
unchanged neural network, and if the comparison result does not exceed a predetermined level, 
eliminating the first synapse, as recited in claim 1 . 

Examiner's response: 

A the applicant acknowledged on page 10, line 18 of the response of August 2, 
2004, the "utility of neural networks is well known" which is conclusory to the cause that 
the "technology is well known" which is exemplified by the text of Mehrotra et al entitlied 
"Elements of Artificial Neural Networks" and cited as prior art in the First Office Action. 
The concept of: 

selecting first and second sending neurons that are connected to the same receiving neuron by 
respective first and second synapses, assuming a correlation of response signals from said first 
and second sending neurons to the same receiving neuron, 

is a description of section of a network that was identified in Mehrotra @ Fig. 1.14. A 

general neuron model can be found in Mehrotra @ Fig. 1.4 with a weighted input 

summation identified in Mehrotra @ Fig. 1.5 and with correlation to synapses identified 

in Mehrotra @ Fig. 1.3. Simply stated, neural networks are massively parallel devices 

wherein as Mehrotra illustrates, in Fig. 1 .14, two neurons are connected to a third 

(receiving) neuron wherein correlation is taking place through the additive function as 

Mehrotra illustrates @ Fig. 1 .5. The concept of: 



interrupting the first synapse and adapting in its place a weight of the second synapse, comparing 
the output signals of the changed neural network with the output signals of the unchanged neural 
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network, and if the comparison result does not exceed a predetermined level, eliminating the first 
synapse 

is a description of pruning such as described in Mehrotra @ para 4.2.1 to include Fig. 
4.6 which is a genetic network pruning algorithm. From a typical neural network such 
as that shown by Mehrotra @ Fig. 1 .14, if a neuron of the perceptron type identified by 
Mehrotra @ fig. 2.5 is removed, and there were only two connection to the receiving 
neuron as the applicant suggests, the only remaining element to the receiving neuron is 
the other perceptron which is influenced by the weight of the second perceptron. 
Returning now to Mehrotra @ Fig. 4.6 and the genetic network pruning algorithm, 
Mehrotra removes the node (neuron - synapse) as long as the removal does not 
penalize performance beyond a desirable tolerance level. Hence, Mehrotra anticipates 
the applicant's claim 1. 



In reference to Applicant's argument: 

The first procedure calls for eliminating connections having weights of small magnitude, as well as 
eliminating their .associated nodes. Presumably, this procedure involves comparing synapse weights 
against a threshold, and eliminating synapses having weights below the threshold. Mehrotra et al„ in 
describing this procedure, do not disclose selecting two sending neurons, assuming a correlation, 
interrupting a synapse and adapting a weight of another synapse, comparing reactions of the network, 
and then eliminating a synapse, as required by claim 1 . In contrast, this procedure merely checks the 
weights of synapses in making a determination of whether to eliminate the synapse. Thus, this procedure 
does not anticipate the invention recited in claim 1 . 

Examiner's response: 

Mehrotra states @ para 4.2.1: "Nodes whose associated connections have small 
magnitude weights may also be pruned." This combined with the Mehrotra genetic 
network pruning algorithm @ Fig. 4.6 is equivalent to eliminating weights below a 



threshold. 
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In reference to Applicant's argument: 

The second procedure calls for determining the effect that a change in the synapse weight has on 
network output, and eliminating synapses if the change is acceptable. The change in synapse weight is 
not disclosed to be based on a weight of another synapse having a different sending neuron and the 
same receiving neuron, as required by claim 1 . Further, no pair of neurons is selected for weight 
adaptation and no correlation assumption is made, also as required by claim 1. Thus, this procedure does 
not anticipate the invention recited in claim 1 . 

Examiner's response: 

The network is made up of a plurality of nodes and when a connection weight is 
changed to zero and the output is observed will of course be comparable to other 
synapses since the process is made in the plurality. The network is made up of a 
plurality of neurons and therefore there is a plurality of pairs of neurons and such pairs 
will be selected for weight adaption. Correlation is by addition and such correlation is 
identified by Mehrotra @ Fig.2.5. Further, since claim 1 "assumes correlation" this 
means that from the perspective of the applicant, correlation may not be there and 
hence correlation is not a limiting criteria by the applicant. 

In reference to Applicant's argument: 

The third procedure calls for pruning input nodes, and does not address identifying and eliminating 
synapses. Thus, this procedure does not anticipate the invention recited in claim 1 . 

Examiner's response: 

Pruning input nodes removes the synapses to other neurons. See Mehrotra @ 
Fig. 1.3. 

In reference to Applicant's argument: 
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The fourth procedure calls for identifying weights to be pruned from a network by examining the second 
derivatives of the error function contained in the Hessian matrix, using a formula to approximate the 
change in output resulting from a small perturbation in a selected weight. Pairs of neurons are not 
selected, weights are not substituted, and actual changes based on the substitution are not made, as 
required by claim 1 . Thus, this procedure does not anticipate the invention recited in claim 1 . 

Examiner's response: 

Indeed all of the applicant's objections are satisfied by Mehrotra. The network is 
made up of a plurality of neurons and consequently there are a plurality of pairs of 
neurons; a plurality of first and second neurons; "pair" is not used by the applicant in 
claim 1. When a neuron is removed, the weight of the nearby neuron takes its place ... 
this is axiomatic in any neural network. Changes based on substitution are made since 
the removal of a neuron causes substitution of a nearby neuron weight ... again it is 
axiomatic. 



In reference to Applicant's argument: 

In summary, Mehrotra et al. describe four pruning procedures for simplifying a neural network. None of 
these procedures includes all of the elements of the method recited in claim 1 . That is, none of the 
disclosed procedures includes the process of selecting first and second sending neurons that are 
connected to the same receiving neuron by respective first and second synapses, assuming a correlation 
of response signals from said first and second sending neurons to the same receiving neuron, interrupting 
the first synapse and adapting in its place a weight of the second synapse, comparing the output signals 
of the changed neural network with the output signals of the unchanged neural network, and if the 
comparison result does not exceed a predetermined level, eliminating the first synapse, as recited in 
claim 1 . 

Examiner's response: 

Above discussions apply. Mehrotra fully anticipates the applicants disclosure. 
Examiner notes that while each point made by the applicant has been answered, it was 
necessary to only show that Mehrotra taught one process that anticipated the 



applicant's invention. 
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In reference to Applicant's argument: 

Claims 2-11 depend from claim 1, and therefore also are not anticipated by Mehrotra et al., for the 
reasons noted with respect to claim 1, and in view of the additional features recited in the dependent 
claims. For example, claim 2 recites that the first and second selected sending neurons are located on the 
same layer. The examiner noted that Mehrotra et al. disclose sending neurons located on the same layer, 
in Fig. 1 .14. Fig. 1 .14 does show that sending neurons exist on the same level, but not that two sending 
neurons selected as part of the process recited in claim 1 are located on the same layer, because that 
process is not disclosed by Mehrotra et al. 

Examiner's response: 

Since Mehrotra does anticipate the applicant's claim 1, the applicant concurs that 
Mehrotra anticipates claims 2-1 1 . 



Examination Considerations 

13. The claims and only the claims form the metes and bounds of the invention. 
"Office personnel are to give the claims their broadest reasonable interpretation in light 
of the supporting disclosure. In re Morris, 127 F.3d 1048, 1054-55, 44USPQ2d 1023, 
1027-28 (Fed. Cir. 1997). Limitations appearing in the specification but not recited in 
the claim are not read into the claim. In re Prater, 415 F.2d, 1393, 1404-05, 162 USPQ 
541, 550-551 (CCPA 1969)" (MPEP p 2100-8, c 2, 1 45-48; p 2100-9, c 1, 1 1-4). The 
Examiner has full latitude to interpret each claim in the broadest reasonable sense. 
Examiner will reference prior art using terminology familiar to one of ordinary skill in the 
art. Such an approach is broad in concept and can be either explicit or implicit in 
meaning. 

14. Examiner's Notes are provided to assist the applicant to better understand the 
nature of the prior art, application of such prior art and, as appropriate, to further 
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indicate other prior art that maybe applied in other office actions. Such comments are 
entirely consistent with the intent and spirit of compact prosecution. However, and 
unless otherwise stated, the Examiner's Notes are not prior art but a link to prior art that 
one of ordinary skill in the art would find inherently appropriate. 
15. Examiner's Opinion: Paras 13. and 14. apply. The Examiner has full latitude to 
interpret each claim in the broadest reasonable sense. 



Conclusion 

1 6. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

1 7. A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
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18. Claims 1-11 are rejected. 



Correspondence Information 



Any inquiry concerning this information or related to the subject disclosure 
should be directed to the Examiner, Joseph P. Hirl, whose telephone number is 
(571 ) 272-3685. The Examiner can be reached on Monday - Thursday from 
6:00 a.m. to 4:30 p.m. 

If attempts to reach the Examiner by telephone are unsuccessful, the 
Examiner's supervisor, Anthony Knight can be reached at (571) 272-3687. 
Any response to this office action should be mailed to: 

Commissioner of Patents and Trademarks, 

Washington, D. C. 20231; 
or faxed to: 

(703) 872-9306 (for formal communications intended for entry); 
or faxed to: 

(571 ) 273-3685 (for informal or draft communications with notation of 
"Proposed" or "Draft" for the desk of the Examiner). ^ 
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Supervisory Patent Examiner 
Group 3600 



